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Abstract 


This paper examines the concepts of decision-making, risk 
analysis, uncertainty and resilience analysis. The relation 
between risk, vulnerability, and resilience is analyzed. The paper 
describes how complexity, uncertainty, and ambiguity are the 
most critical factors in the definition of the approach and criteria 
for decision-making. Uncertainty in its various forms is what 
limits our ability to offer definitive answers to questions about the 
outcomes of alternatives in a decision-making process. It is shown 
that, although resilience-informed decision-making would seem 
fundamentally different from risk-informed decision-making, this 
is not the case as resilience-analysis can be easily incorporated 
within existing analytic-deliberative decision-making 
frameworks. 
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1. Introduction 


Decision-making is the process of choosing among alternatives. The nature and context of decisions 
can vary and there can be a variety of criteria to be considered in making a choice. The alternatives can 
involve a large number and different levels of uncertainties about the factors that influence the ultimate 
outcomes. For high-consequence decisions with potential outcomes that can have a large impact on valued 
things, such as human life, property, wealth, and the environment, the decision-making process should 
involve consideration of uncertainties about the outcomes. 

This paper presents an examination of the high-stakes decision-making process with consideration of 
the uncertainties associated with the alternatives. The next section presents a brief overview of the concept 
of decision-making. This is followed by a review of risk-informed decision-making, in which potential 
negative outcomes are considered in the selection of an alternative. In situations of high consequence and 
uncertainty, the robustness of the alternatives in the sense of insensitivity to uncertainties may be an 
advantageous consideration to increase confidence in the expected outcome of the selected alternative. A 
simple framework for resilience-informed decision-making is proposed to complement a risk-informed 
decision-making process. A summary and final remarks conclude this article. 


2. Decision-Making 

A decision is a resolution on an issue or a problem of interest being considered. The process of making 
a decision involves three elements: values, alternatives, and facts (Buede, 2009). Values are the criteria 
used to assess the utility of the outcome. Depending on the context, values can be objective or subjective, 
and quantitative or qualitative. The values capture the needs, objectives, and preferences of the 
stakeholders, which are the people and organizations that have an interest and can influence or be influenced 
by a decision. The alternatives may be given or can be the result of a creative process to generate potential 
solutions to the problem of interest. The facts are everything known about the alternatives and the context 
or environment in which the alternatives will be deployed or applied. Facts are the relevant data, 
information, and knowledge used in the process of assessing the alternatives against the values in order to 
make the decision. The decision makers use available facts, history, experience, and judgment to select an 
alternative. 

NASA’s Systems Engineering Flandbook (NASA, 2007) describes a generic decision-making process. 
The essential steps are as follows: 

1 . Define the criteria for evaluating alternative solutions 

2. Identify alternative solutions to address decision issues 

3. Select evaluation methods and tools 

4. Evaluate alternative solutions with the established criteria and selected methods 

5. Select recommended solutions from the alternatives based on the evaluation criteria 

6. Report analysis results with recommendations, impacts, and corrective actions 

In general, decision-making is an iterative analytic-deliberative process consisting of a technical sub- 
process to analyze the problem and the alternatives, and a deliberation sub-process to assess the alternatives 
and inform the analysis. The analysis and deliberation processes are iterated with information flowing 
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between them until a final decision is made (Ersdal & Aven, 2008) (Zio & Pedroni, 2012). 

An important consideration in the definition of a decision-making process is the cost and time required 
to implement the process. Generally, a formal decision-making process is used only for decisions 
involving: 

• High stakes (i.e., the outcome can have a significant impact on cost, safety, etc.), 

• Complexity (i.e., outcomes are difficult to understand), 

• Uncertainty (i.e., inadequate knowledge to assess and rank alternatives with high confidence in the 
outcome), 

• Multiple criteria (i.e., formal analysis methods may be needed to resolve conflicts among 
evaluation criteria), and 

• Diversity of stakeholders (i.e., stakeholders may have different and conflicting values and 
preferences) (NASA, 2007) (Klinke & Renn, 2002). 

A formal decision-making process must be justified by a cost-benefit argument that considers the potential 
impact of the outcome and the resources and time needed to implement the process. 

Hester et al. (2012) have proposed a system for classifying stakeholders and determining an appropriate 
level of engagement in making decisions about a system. The classification attributes include: 

• Power (i.e., ability to influence other stakeholders), 

• Legitimacy (i.e., actions are proper or desirable within some socially constructed system of norms, 
beliefs, or definitions), and 

• Urgency (i.e., degree to which stakeholder’s views require immediate action). 

Additionally, Hester et al. consider the level of support of particular stakeholders for system-related 
decisions, with the categories of supportive, mixed, non-supportive, and marginal. Based on the patterns 
of degrees of these attributes, Hester et al. propose a set of strategies for engagement and involvement of 
the stakeholders in the decision-making process. These strategies include: involve, collaborate, defend, 
monitor, and no action. 

A number of techniques are available for individual and group decision-making. Techniques for 
individual decision-making include: 

• Prioritization (i.e., rank by order of importance), 

• Satisficing (i.e., heuristic of selecting the first alternative that meets minimum requirements), 

• Opportunity cost (i.e., the benefit received from selecting a different alternative), 

• Bureaucratic (i.e., structured and objective criteria-based), and 
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• Elimination (i.e., process of comparison and elimination of alternatives that do not meet selected 
criteria and continuing until only one alternative remains). 

Group decision-making techniques include: 

• Consensus (i.e., majority decision with minority support to go along), 

• Voting (e.g., majority or plurality), and 

• Delphi method (Helmer, 1967). 

A major concern and source of difficulty in high-consequence decision-making is the presence of 
uncertainty about the alternatives. The causes of uncertainty include: 

• Lack of information or knowledge, 

• Abundance of information or knowledge, 

• Conflicting nature of pieces of information, 

• Measurement errors, linguistic ambiguity, and 

• Subjectivity of analysis opinions (Zio & Pedroni, 2012). 

Uncertainties can also be classified as: 

• Knowledge uncertainty (i.e., only sparse statistics or random data are available), 

• Modeling uncertainty (i.e., incomplete or inaccurate models), and 

• Limited predictability or unpredictability (i.e., outcome is highly sensitive to initial conditions) 
(Aven T. , 2006). 

fundamentally, decision-making uncertainties are either aleatory (i.e., due to inherently random processes) 
or epistemic (i.e., due to lack of knowledge about properties and conditions of phenomena that determine 
the behavior of a system) (Zio & Pedroni, 2012). Epistemic uncertainty can be reduced by acquiring 
knowledge, but aleatory uncertainty is irreducible. Uncertainty is a fundamental limitation on the 
effectiveness of a decision-making process. 


3. Risk Analysis 

Risk has been defined as “the potential for an unwanted outcome resulting from an incident, event, or 
occurrence as determined by its likelihood and the associated consequences” (DHS, 2010). However, there 
are multiple perspectives on the concept of risk (Aven T. , 2006). One definition of risk is the combination 
of probabilities and consequences, where the consequences are assessed in terms of valued things such as 
health, the environment, and safety. Another definition of risk is the expected value of consequences given 
by the product of probabilities and consequences. Risk has also been defined as a combination of the 
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elements of events, consequences, probabilities, and uncertainties. In the most general approach, these 
elements are kept and handled separate, as there is no agreement on a reducing mathematical formulation 
that is universally suitable for all decision situations (Aven T. , 2010). Other definitions of risk include the 
“effect of uncertainty on objectives” and “a situation or event where something of human value (including 
humans themselves) has been put at stake and where outcome is uncertain” (Aven T. , 2010). 

There are seven domains where risk is a major consideration: 

• Environmental risk (i.e., related to events such as floods and pollution), 

• Lifestyle risk (i.e., related to food, drugs, driving, and so on), 

• Medical risk (i.e., related to medical treatments, surgery, etc.), 

• Interpersonal risk (i.e., social relationships, parenting, and others), 

• Economic risk (i.e., such as investment, employment decisions and events), 

• Criminal risk (i.e., being a victim of illegal activity), and 

• Technical risk (i.e., accidental side effects of technological innovations) (Johansen, 2010). 

In all domains, the focus is on likelihood and severity of future consequences of present decisions and 
actions. 

One view on the purpose of risk analysis is “to understand a risk in order to do something about it” 
(Johansen, 2010). However, at a more basic level, risk reduction is not the primary goal of a risk 
assessment. Risk analysis serves to provide input to a particular decision-making process with a wider 
range of additional considerations (Johansen, 2010). Thus, risk assessment is a tool to generate information 
that contributes to reducing the uncertainty of decision makers about the outcome of decision alternatives. 

Risk analysis can be based on deterministic and probabilistic analysis methods (Kirchsteiger, 1999). 
Deterministic risk analysis is based on worst-case assumptions that bound the uncertainties on likelihood 
and severity of consequences. This effectively simplifies the analysis and provides reasonable assurance 
that the risk level is acceptable. A problem with this approach is that the definition of worst-case conditions 
is subjective and potentially arbitrary, which can lead to consideration of scenarios which are possible but 
highly unlikely. Deterministic risk analysis is a conservative approach that offers only broad (and not deep) 
insight into the likelihood of possible outcomes. Probabilistic risk analysis (PRA) is intended to be more 
obj ective, detailed, and quantitative and offer deeper insight into the relative likelihood of outcomes and 
the implications of uncertainty. NASA has published a comprehensive guide on PRAs (NASA, 2011). 
PRA is centered on the quantification of risk metrics, which are probabilistic performance measures that 
appear in decision models. Examples of risk metrics are the probabilities of fuel ta nk explosion and of 
system failure. Such metrics and the techniques used in their calculations can offer insight into the relative 
importance of various sources of uncertainty and serve as a reference in optimal resource allocation 
decisions. In general, the output of a risk analysis consists of undesirable risk-significant scenarios, the 
consequences of those scenarios, the probabilities, and the uncertainty or confidence in the results of the 
analysis (NASA, 2011). 

Various processes have been proposed for risk analysis (NASA, 2010) (NASA, 2011) (Johansen, 2010) 
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(Biringer, Vugrin, & Warren, 2013). The basic steps are: 

1 . Define the system, 

2. Identify the alternatives, 

3. Identify the hazards (i.e., what can go wrong), 

4. Estimate the likelihood and consequences for each hazard, and 

5. Combine the results into a single comprehensive risk picture. 

A wide range of analysis tools and techniques are available, including fault tree analysis; failure mode and 
effects analysis; and modeling and simulation. The risk analysis results are used in risk assessments where 
the goal is to assess the confidence in the analysis results and decide whether the level of risk is acceptable, 
tolerable (i.e., operationally manageable), or unacceptable. 

The effectiveness of risk assessments is complicated by three major factors that limit the accuracy of 
the results (Ben-Haim, 2012). These include our evolving understanding of the world and its uncertainties, 
the inherent random nature of physical phenomena, and the fact that learning from experience involves 
untestable assumptions in rational processes of inductive reasoning about the physical world. 
Considerations of ontological uncertainties (i.e., what exists and its nature) and epistemological 
uncertainties (i.e., acquisition and thoroughness of knowledge about what exists) complicate the assessment 
of risk as it is clear that truly objective and accurate assessment of risk is not possible, except for the simplest 
of situations. Situations involving complexity, uncertainty, and ambiguity necessarily require subjective 
judgments and considerations of stakeholder values and preferences to arrive at a decision. 


4. Risk-Informed Decision-Making 

There are two basic approaches for using risk information in decision-making: risk-based and risk- 
informed (Johansen, 2010). Risk- based decision-making considers only risk assessment results to make 
decisions. This approach may be adequate for simple systems when the stakeholders are aligned in their 
values and preferences. Safety-critical decision-making processes may follow this approach. On the other 
hand, risk-informed decision-making (RIDM) considers risk along with other sources of information, 
such as cost-benefit analyses and laws and regulations, to compare and deliberate on the preferred 
alternative. The basic steps of RIDM are: 

1 . Selection of alternatives, 

2. Risk analysis for each alternative, and 

3. Alternative selection (Zio & Pedroni, 2012) (NASA, 2010). 

Note that RIDM is an iterative analytic and deliberative process in which information can flow both ways 
between analysis and deliberation to ensure that alternative generation, risk analysis, and deliberation for 
alternative selection are adequate and as detailed and complete as needed to ensure an explicit and 
defensible decision. RIDM is the preferred approach when there is a multitude of stakeholder values and 
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preferences (i.e., ambiguity), and there is a need to find the best alternative that satisfies multiple objective 
and subjective criteria. Both deterministic and probabilistic analysis results may be use in R1DM. Large 
groups of stakeholders from diverse segments of society may participate in the generation of alternatives 
and the process of deliberation (NRC, 1996). In such situations, risks may be traded against other attributes 
in order to ensure selection of the best overall alternative (NASA, 2007) (Amendolla, 2001). 

The research literature on risk analysis and risk-informed decision-making is extensive. The principal 
issues are the disambiguation and definition of concepts, dealing with uncertainty, strategies for dealing 
with risk, and communicating risk to the decision makers 1 . Risk and R1DM are very complex topics and 
there is not a single solution that fits all possible situations. Instead, each situation requires consideration 
of the specific circumstances and framing the problems and approach to ensure adequate coverage of 
relevant factors. Complexity, uncertainty, and ambiguity are the major categories of consideration in 
choosing a R1DM approach (Amendolla, 200 1 ). 

Ersdal and Aven (2008) have identified five common decision principles used in R1DM: 

■ Cost-benefit analysis (i.e., choose the alternative that maximizes the total benefit taking into account 
costs, possible losses due to negative outcomes, and any expected income or benefit); 

■ Risk acceptance criteria (i.e., the risk is compared to a threshold level of acceptability); 

■ Cautionary and precautionary principle (i.e., seek alternatives that are robust or insensitive to 
uncertainties in the likelihood of severe outcomes by avoiding those outcomes and reducing their 
impact should the undesired outcomes occur); 

■ As Low As Reasonably Practicable (ALARP) principle (i.e., all available risk reduction (likelihood, 
severity, and uncertainty) should be implemented unless gross disproportion between cost and benefit 
can be demonstrated); and 

■ Multi-attribute analysis (i.e., where multiple criteria, including risk-related ethical factors, must be 
considered in deliberations to identify a suitable and balanced alternative). 

Multi-attribute decision-making is the most general situation. The decision principle applied depends on 
ethical, safety, and economic considerations, as well as the risk posture of the stakeholders (i.e., the 
acceptable level of risk for a particular potential level of reward). 

Decisions with high uncertainty and potentially high negative consequences require special 
consideration. In these situations, it may be the judgment of experts and scientists that the likelihood of 
strongly negative consequences is not insignificant, but the effectiveness of a risk analysis is limited by a 
high level of epistemic and aleatory uncertainty. Ethical considerations may then be strong determinants 
of the proper course of action (Aven T. , 2006). In some situations, the major uncertainty is in the likelihood 
of the hazards. This is the case in infrastructure risk assessments where the threats can be random or 
deliberate with the range of threats including natural events, accidents, and malevolent actions (Biringer, 
Vugrin, & Warren, 2013). In this case, it may be advantageous to avoid direct estimation of the likelihood 
of hazards and instead consider assessments of the overall threat environment (i.e., possibilities, 
capabilities, motive, etc.) (Haimes, 2009). Application of the cautionary and precautionary principle then 
leads to consideration of alternatives aimed at avoiding hazards and mitigating the effects of hazards that 


1 The Bibliography lists many relevant references on these issues. 
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may occur (J ohansen, 2010). In effect, the desirable alternative is one that offers a high degree of robustness 
(i.e., insensitivity), and thus, assurance against uncertainties. 


5. From Risk to Resilience 

The concept of risk is not adequate if the likelihood of hazards is highly uncertain. In that case, the 
emphasis should be on the analysis of the consequences of potential hazards. For this, risk is decomposed 
into hazards and vulnerability (Ciurean, Schroter, & Glade, 2013): 

Risk = Hazard x Vulnerability 

A hazard is a “dangerous phenomenon, substance, human activity, or condition that may cause loss of 
life, injury or other health impacts, property damage, loss of livelihoods and services, social and economic 
disruption, or environmental damage” (Ciurean, Schroter, & Glade, 2013). In addition, Ciurean et al. 
(2013) state that “Within the risk management framework, vulnerability pertains to consequence analysis” 
and “defines the potential for loss to the elements at risk caused by the occurrence of a hazard.” 
Furthermore, Aven (2011) has referred to vulnerability as the antonym of robustness. According to 
Johansen (2010): “Regardless of the improbability of an event, unfortunate outcomes may still occur. 
Reducing vulnerability on the other hand, will always attenuate risk through reduced consequence severity. 
Especially important in this are cases of great epistemic uncertainty.” Johansen describes vulnerability as 
composed of three elements: 

• Exposure (i.e., effectiveness of protection against initiation of disruption due to hazards), 

• Sensitivity (i.e., degradation in performance due to propagation of hazard effects), and 

• Resilience (i.e., ability to bounce back after a disturbance). 

Robust mitigation of hazard effects requires a balanced allocation of resources to minimize vulnerability in 
a cost-effective manner. All of these factors would be considered in a comprehensive analytic-deliberative 
decision-making process for high consequence situation. 

Francis and Bekera (2014) proposed a resilience concept that encompasses three capacities: 

• Absorptive (i.e., ability to absorb and contain the effects of hazards), 

• Adaptive (i.e., the ability to internally adapt to compensate for exceedance of absorptive capacity), 
and 

• Recovery/restorative (i.e., ability to return to normal or improved level of performance). 

Many other definitions of resilience have been proposed in the literature. Biringer et al. (2013) list 23 
different definitions proposed by different authors. Haimes (2009) defined resilience as “the ability of the 
system to withstand a major disruption within acceptable degradation parameters and to recover within 
acceptable time and composite costs and risks.” Aven (2011) has also examined some existing definitions 
of vulnerability and resilience in the literature and proposed that vulnerability is assessed with respect to a 
fixed set of hazard events, while resilience is assessed relative to any set of hazards. From this perspective, 
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vulnerability is a special case of resilience. Biringer et al.’s decomposition seems better aligned with the 
common sense of the notion of resilience as the ability to cope with change, including resisting degradation 
and recovering in a timely manner. From this perspective, Biringer et al.’s concept of protection aligns 
with Johansen’s concept of exposure, and Biringer et al.’s resilience would include the concepts of 
sensitivity and resilience in the sense described by Johansen (2010). That is: 

Vulnerability = Protection x Resilience 


and 


Risk = Hazard x Protection x Resilience. 


6. Resilience-Informed Decision-Making 

Risk analysis considers three questions: What can go wrong? How likely is it? What are the associated 
consequences? (NASA, 2011). A risk analysis should also consider the uncertainty. Thus, the risk analysis 
elements are: 

• Scenarios, 

• Likelihood, 

• Consequences, and 

• Uncertainty. 

From this perspective, resilience analysis should be an explicit and integral element of the analysis of 
consequences, and the results of these analyses should be included in the considerations of risk-informed 
decision-making. Alternatively, a dedicated resilience analysis would consider a wide range of potential 
hazards to assess the capability of the system to absorb, adapt, and recover. Resilience analysis would be 
similar to robustness and sensitivity analyses and have the goal of determining the range of possible effects 
under conditions of high uncertainty. 

A resilience-informed decision-making process would not be significantly different from a risk- 
informed decision-making process. Both approaches acknowledge that decision-making for high 
consequence situations requires consideration of technical as well as social factors such as the values and 
preferences of the stakeholders, especially to define the decision criteria and the acceptable level of 
uncertainty in the outcome of decisions. 

Figure 1 illustrates a generic informed decision-making framework that includes various types of 
analyses to generate critical information for consideration in the process of stakeholder deliberation. This 
framework is based on the one presented by Johansen (2010), and it is similar to the Risk-Informed 
Decision-Framework used by NASA and the US Nuclear Regulatory Commission (USNRC) (Zio & 
Pedroni, 2012) (NASA, 2010) (NASA, 2011) (NASA, 2007) (Johansen, 2010). 




Figure 1: General Analytic-Deliberative Decision-Making Framework (adapted from (Johansen, 2010)) 


7. Final Remarks 

This paper has presented an examination of the concepts of decision-making, risk analysis, uncertainty, 
and resilience analysis. The relation between risk, vulnerability, and resilience was analyzed. In addition, 
a resilience-informed decision-making approach that builds on existing informed decision-making 
frameworks was proposed. 

This paper provides insight into the breadth and depth of available published work produced by risk 
researchers worldwide. Additionally, the paper shows that, although resilience-informed decision-making 
would seem fundamentally different from risk-based decision-making, this is not the case as resilience- 
analysis can be easily incorporated within existing analytic-deliberative decision-making frameworks. The 
paper also describes how complexity, uncertainty, and ambiguity are the most critical factors in the 
definition of an approach and criteria for decision-making. The paper also offers insight into the approaches 
for structuring, analyzing, and managing uncertainty. Ultimately, uncertainty in its various forms is what 
limits our ability to offer definitive answers to questions about the outcomes of alternatives in a decision- 
making process. 
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